1 2 Accelerating the discontinuous Galerkin method for seismic wave 3 propagation simulations using multiple GPUs with CUDA 4 and MPI 5

8 Abstract We have successfully ported an arbitrary high9 order discontinuous Galerkin method for solving the three10 dimensional isotropic elastic wave equation on unstruc11 tured tetrahedral meshes to multiple GPU using CUDA and 12 MPI and obtained a speedup factor of about 28.3 for the 13 single-precision version of our codes and a speedup factor 14 of about 14.9 for the double-precision version. The GPU 15 used in the comparisons is NVIDIA Tesla C2070 Fermi, 16 and the CPU used is Intel Xeon W5660. To effectively 17 overlap inter-process communication with computation, we 18 separate the elements on each subdomain into inner and 19 outer elements and complete the computation on outer 20 elements and fill the MPI buffer first. While the MPI 21 messages travel across the network, the GPU performs 22 computation on inner elements, and all other calculations 23 that do not use information of outer elements from neigh24 boring subdomains. A significant portion of the speedup 25 also comes from a customized matrix–matrix multiplica26 tion kernel, which is used extensively throughout our 27 program. Preliminary performance analysis on our parallel 28 GPU codes shows favorable strong and weak scalabilities. 29 30